Publication Details
Andriy Kot, Andrey Chernikov and Nikos Chrisochoides.
Published in IEEE International Parallel and Distributed Processing Symposium, pages 164 -- 175, May, 2011
Abstract
We present an out-of-core run-
time system that supports effective parallel
computation of large irregular and adaptive
problems, in particular
unstructured mesh generation (PUMG). PUMG is a
highly challenging application due to intensive
memory accesses,
unpredictable communication patterns, and variable
and irregular data
dependencies reflecting the unstructured spatial
connectivity of mesh
elements.
Our runtime system allows to
transform the footprint of parallel applications
from wide and shallow into narrow and deep by
extending the memory utilization
to the out-of-core level. It simplifies and
streamlines the development of
otherwise highly time consuming out-of-core
applications as well as the
converting of existing applications. It utilizes
disk, network and memory
hierarchy to achieve high utilization of computing
resources without sacrificing
performance with PUMG. The runtime system combines
different programming
paradigms: multi-threading within the nodes using
industrial strength software
framework, one-sided active messages among the
nodes, and an out-of-core subsystem
for managing large datasets.
We performed an evaluation on traditional parallel platforms to stress test all layers of the run-time system using three different PUMG methods with significantly varying communication and synchronization patterns. We demonstrated high overlap in computation, communication, and disk I/O which results in good performance when computing large out-of-core problems. The runtime system adds very small overhead (up to 18% on most configurations) when computing in-core which means performance is not compromised.
